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Executive Summary 


At the request of the Chief Academic Officer, the Office of Shared Accountability conducted a study 
of successful secondary students in Montgomery County Public Schools (MCPS) and what helped 
them to achieve academic success. The study first examined performance at the advanced level 
over time. In response to the finding that Black or African American and Hispanic/Latino students 
were less likely to stay at an advanced level of achievement from Grade 8 to 12, the study identified 
curricular options and high school programs that supported advanced performance by these students 
as they went on to complete high school. The specific research questions were as follows: 

1. What is the level of achievement at the advanced level among secondary students over time? 
Specifically, what percentage of students performed at an advanced level in Grade 8 and then 
continued as high achievers at the end of high school? Which students were most likely to stay 
at the advanced level over time, from Grade 8 to the end of high school? 

2. Which high school programs or curricular options were most promising in supporting Black 
or African American and Hispanic/Latino students to continue achieving at an advanced level 
at the end of high school? 

3. With respect to the programs and curricular options identified in question 2, what are the 
characteristics or components of each one? What are common aspects across them? 


Summary of Methodology 


For question 1, the study sample was all students who completed Grade 8 in MCPS in 2008-2009. 
For questions 2 and 3, the sample was limited to those Black or African American and 
Hispanic/Latino students who were high achievers in Grade 8 and also completed Grade 12 in 
MCPS. Measures of high achievement in Grade 8 were advanced level on the Maryland School 
Assessments in mathematics; grade of B or higher in a mathematics course (Algebra I or higher); 
and a score of 231 or higher on Measures of Academic Progress assessments in reading. Measures 
of high achievement for the end of high school were a score of 1650 or higher (out of 2400) on the 
SAT or a score of 24 or higher on the ACT; a score of 3 or higher on an Advanced Placement (AP) 
exam or a score of 4 or higher on an International Baccalaureate (IB) exam; and completion of a 
rigorous high school program as defined by the Maryland State Department of Education. In each 
grade level, students had to meet only one of the measures to be classified as high achievers for this 
study. 


Participation in curricular options that support advanced learning (i.e., Advanced-level, AP, Honors, 
or IB courses) was based on course completion records. Program codes from the Online 
Administrative Student Information System (OASIS) database were used to identify participants in 
choice, magnet, or signature programs that support advanced learning. Because the data in OASIS 
was incomplete, additional approaches (e.g., analysis of course completion) were used, but it was 
not possible to identify participants for all programs. Analyses included descriptive statistics for all 
questions plus measures of statistical and practical significance where appropriate. 


Summary of Findings 


Question I. Nearly one out of six (59%) students in Grade 8 performed at an advanced level. 
Among these advanced eighth graders who were still enrolled in MCPS for Grade 12, more than 
three quarters (76%) continued to perform at an advanced level at the end of high school. Several 
subgroups were more likely to stay at the advanced level from Grade 8 to 12: females, Asian or 
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White students, students who had never received English for Speakers of Other Languages, students 
who had never received Free and Reduced-price Meals System services, and non-recipients of 
special education services. 


Question 2. Ten programs and curricular options (some of which overlapped) were 
identified as the most promising in supporting high-achieving, Grade 8 Black or African American 
and Hispanic/Latino students to continue advanced achievement. They included four programs in 
which all or nearly all participants finished high school at an advanced level: 

e CAP at Montgomery Blair HS 

e Global Ecology or Humanities House at Poolesville HS 

e IB at Richard Montgomery HS 

e Science, Mathematics, & Computer Science at Montgomery Blair HS and Poolesville HS 


The remaining six promising programs and curricular options were those with the greatest impact 
(as measured by effect sizes of practical significance) on supporting high-achieving, Grade 8 Black 
or African American and Hispanic/Latino students to continue as high achievers at the end of high 
school: 

e Six or more semesters of Advanced-level courses 
Six or more semesters of AP courses 
Four or more semesters of IB courses 
Local IB programs at seven high schools 
Signature programs that require AP courses at four high schools (i.e., APEX, APPS, 
Humanities & Arts, PEAC) 
e Ulysses Signature Program at Northwest HS 


Question 3. Descriptions of each promising program and curricular option are in 
Appendix B. Each of them required rigorous courses; 6 of the 10 required applications; and 4 or 5 
required a research project during the student’s senior year. 


Recommendations 


The report’s findings suggest the following recommendations to support continued achievement of 
Black or African American and Hispanic/Latino students: 

e Institute districtwide strategies that, starting in middle school, encourage high-achieving 
Black or African American and Hispanic/Latino students to enroll in IB programs; signature 
programs that require AP courses; or at least two, year-long Advanced-level, AP, or IB 
courses. Augment course bulletins to indicate which courses are Advanced-level. 

e Consider establishing a signature program or modify an existing signature program to 
require AP courses at the 10 high schools without such a program or an IB program. 

e Consider expanding the availability of Advanced-level, AP, and IB courses and the size of 
IB and signature programs to support increased enrollment in them. 


The difficulty in identifying student participation in programs limited the study’s analyses and 
findings. The final recommendations on future research include one to avoid these difficulties: 
e Develop a monitoring system to ensure that staff members at every school consistently and 
completely enter program codes for participants for every year of the program. 
e Confirm the findings by replicating the research with a more recent class of MCPS graduates. 


Program Evaluation Unit Vi Achievement at an Advanced Level, Secondary 


Achievement at an Advanced Level Among Secondary Students: 
Changes Over Time and Options that Support Black or African American 
and Hispanic/Latino Students 


Problem Statement and Rationale 


The mission of Montgomery County Public Schools (MCPS) is to ensure every student has the 
academic, creative problem solving, and social emotional skills they need to be successful in 
college and careers as reflected in the district’s Strategic Planning Framework—Building Our 
Future Together: Students, Staff and Community (MCPS, 2015a). This framework makes clear 
the district’s commitment to education, such that “academic success is not predictable by race, 
ethnicity, gender, socioeconomic status, language proficiency, or disability.” 


Although MCPS has narrowed the gap in performance among racial subgroups in some areas, 
including graduation rate, dropout rate, and access to rigorous classes, gaps in achievement remain 
in other areas and, in some cases, have increased (MCPS, 2015a). For example, with respect to 
reading achievement in Grade 8 in 2013-2014, the majority of students in every subgroup, except 
those receiving English language or special education services, met or exceeded the benchmark 
for proficiency (Figure 1). The largest gap among racial/ethnic groups was 26 percentage points 
between Hispanic/Latino and White students. However, for reading at the advanced level, less 
than one half of most subgroups met this benchmark; further, the largest gap among racial/ethnic 
groups, again between Hispanic/Latino and White students, was 43 percentage points (Figure 1). 


Clearly there are differences in the percentages of students who reach various educational 
milestones, but it is worth noting that there are students from every subgroup who do attain those 
milestones. Therefore, at the request of the Chief Academic Officer, the Office of Shared 
Accountability conducted this study of successful students and what helped them to achieve 
academic success. The study first examined performance at the advanced level over time, 
including analysis by subgroup. In response to these findings, the second part of the study focused 
on identifying those MCPS programs and curricular options that supported advanced levels of 
performance by Black or African American and Hispanic/Latino students as they went on to 
complete high school. Specifically, the focus was programs and other curricular options that 
MCPS provides to students who have the ability, motivation, or potential to perform at high levels. 
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Figure 1. Percentages of Grade 8 students reading at the proficient and advanced levels in 2013-2014 for total and for subgroups. Source MCPS (2015a). 
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Background 
Programs and Curricular Options for Advanced Level Learning in High School 


MCPS supports high school students with the ability, motivation, or potential to perform at high 
levels through programs, coursework, and other curricular options, as follows (MCPS, 2015b). 


Programs. A program refers to a set of required and elective courses designed to meet 
specific academic goals. Choice options for high school (HS) programs that support advanced 
level learning include (MCPS, 2015b & 2015c): 


e Programs that require applications (and may have a limited number of openings) and are 
open to all MCPS high school students: 
o Global Ecology House: Poolesville HS 
o International Baccalaureate Magnet Program (IB): Richard Montgomery HS 
o Technology: Thomas Edison High School of Technology 
o Visual Art Center: Albert Einstein HS 


e Programs that require applications and are limited to students eligible to attend certain 
MCPS high schools: 
o Humanities House: Poolesville HS 
o Science, Mathematics, and Computer Science: Montgomery Blair HS and 
Poolesville HS 


e Programs that require applications and are limited to Downcounty Consortium (DCC) 
residents or students attending a DCC middle school: 
o Biomedical Magnet Program: Wheaton HS 
o Communication Arts Program (CAP): Montgomery Blair HS 
o Engineering Magnet Program: Wheaton HS 
o Leadership Training Institute: John F. Kennedy HS 


e Gifted and talented/learning disabled programs that require placement as a result of the 
Individual Education Program process and are open to all MCPS high school students: 
o Walter Johnson HS 
o Northwood HS 
o Watkins Mill HS 


e Local IB programs that are limited to students enrolled at the following high schools: 


o Bethesda-Chevy Chase HS o Seneca Valley HS 
o Albert Einstein HS o Springbrook HS 
o John F. Kennedy HS o Watkins Mill HS 
o Rockville HS 


Coursework. Options for advanced level learning include Honors, Advanced Placement 
(AP), and Advanced-level courses, all of which provide rigorous and challenging studies (MCPS, 
2015d). A variety of Honors and AP courses in multiple content areas are available at each MCPS 
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high school to any student who demonstrates the motivation or potential to perform at high levels. 
Each Honors course includes curriculum adaptations for accelerated and enriched learning. AP 
courses are those courses for which a College Board Advanced Placement examination exists. 
Advanced-level courses are based upon previous achievement in a sequence of study; these courses 
are available in several content areas, including information technology/computer science, foreign 
languages, mathematics, and science. Students may enroll in individual Honors, AP, or Advanced- 
level courses; there is not a requirement to enroll in an Honors, AP or Advanced-level program. 
IB courses also provide rigorous and challenging studies, both to students enrolled in this program 
and students who take individual IB courses, without enrolling in the full program. (The option to 
take individual IB courses is available at six of the eight high schools that offer the IB program.) 


Other curricular option. Another advanced level curricular option for MCPS high school 
students is dual enrollment, which is completing college or university courses while enrolled in 
high school. However, this option is dependent on the student’s grade point average (GPA). 


Signature Programs 


Many MCPS high schools offer signature programs that “integrate a specific focus or 
distinguishing theme with the skills, concepts, and instructional strategies of some portion of a 
school’s curriculum” and are limited to students enrolled at the school (MCPS 2015e). The 
following signature programs require applications; and all, except for the two at Churchill HS, also 
require enrollment in AP or honors classes!: 


e Advanced Placement Experience Scholars (APEX) Program: Walter Johnson HS 

e Advanced Placement Power Scholars (APPS): Clarksburg HS 

e Creative & Performing Arts and Mathematics, Technology, & Science: Winston Churchill 
HS 

e Humanities & Arts: Wootton HS 

e Pursuing Excellence in the Academic Curricula (PEAC) Scholars: Col. Zadok Magruder HS 

e Ulysses Program: Northwest HS 


' The AP Scholar Program at Quince Orchard High School that awards certificates at the end of high school was 
excluded because students do not join the program earlier in high school. 


Program Evaluation Unit + Achievement at an Advanced Level, Secondary 


Literature Review 


This review focuses on studies of changes in advanced level performance over time and of 
advanced level performance for Black or African American and Hispanic/Latino students. 


Changes in Performance Over Time for Advanced-level Students 


National studies. In a study focused on high-achieving students from low-income families. 
Wyner, Bridgeland, and Diiulio (2007) defined high achievers as those who scored in the top 
quarter on a nationally normed standardized test and low income as a family income below the 
national median. As first graders, these high-achieving students were demographically (other than 
income) and geographically very similar to all U.S. first graders. However, based on a longitudinal 
study, more of these high-achieving students from low-income families fell out of the high 
achievement group in elementary and high school than their higher income peers. 


In a comprehensive study of differences in achievement between subgroups at the highest levels 
of student performance, Plucker, Burroughs, and Song, labeled these differences “excellence 
gaps” (2010, p. 2). These authors used the National Assessment of Educational Progress (NAEP) 
program that reports student performance in four categories, including advanced. Based on results 
for Grades 4 and 8 from 1998 to 2007, they identified excellence gaps in every year for both 
mathematics and reading, with lower performance by Blacks vs. Whites, and Hispanics vs. Whites, 
and also for English Language Learners (ELLs) vs. non-ELLs, and students eligible for Free and 
Reduced-price Meals System (FARMS) vs. non-eligible students. These gaps persisted or widened 
over the years under study. Other studies also found that gaps increased over time more for 
advanced students than for students with lower levels of achievement. 


Plucker, Hardesty, and Burroughs (2013a) updated their earlier study with data from 2011. They 
found that the excellence gaps had increased among racial/ethnic groups, including gaps in favor 
of Asian/Pacific Islander students compared to Black, Hispanic, and White students, along with 
gaps between ELLs vs. non-ELLs and recipients vs. non-recipients of FARMS. Excellence gaps 
widened because there were increases in the percentages of the Asian/Pacific Islander, White, non- 
FARMS, and non-ELL students who scored at the advanced level, especially in mathematics, 
while, for the other groups, the percentage of students scoring at the advanced level was essentially 
stable. Based on NAEP scores for Grades 4 and 8, they concluded that progress within the country 
in closing achievement gaps at the proficient level was unrelated to changes in excellence gaps. 
Likewise, Bidwell (2013) presented evidence that achievement gaps among students who perform 
at an advanced level actually increased during the No Child Left Behind era, when policy 
incentives and disincentives focused on improving achievement of students performing at below 
grade level. 


Xiang, Dahlin, Cronin, Theaker, and Durant (2011) studied individual, high-achieving students, 
defined as those who scored at or above the 90" normed percentile on the Measures of Academic 
Progress assessments in mathematics (MAP-M) or reading (MAP-R). (Their sample came from a 
database of 4,800 public school systems.) Like the proposed study, these authors looked at students 
at two points in time; specifically, they examined one cohort in Grade 3 and again in Grade 8 and 
a second cohort in Grade 6 and again in Grade 10. A majority of high achievers in both cohorts 
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(52-70%) maintained their status; for example, 56% of high achievers in Grade 6 reading were 
still high achievers in reading at Grade 8. Due to other students moving to the high achievement 
level, the percentage of high achievers in each subject grew slightly for both cohorts. Movement 
into and out of the high achievement level was concentrated among students with scores between 
the 70" and 90" percentiles; in other words, few students moved from low levels of achievement 
to high levels. The proportions of African American, Hispanic, and Native American students in 
high achieving groups were stable over time (7-9%). 


State studies. Research about students in a variety of states reflect the trend in national 
studies that gaps increased over time more for advanced students than for students with lower 
levels of achievement. 


In a longitudinal study of students in Texas public schools, Hanushek and Rivkin (2006) divided 
third graders into groups, based on reading levels and then examined the changes in the Black— 
White achievement gap from Grade 5 to Grade 8 for each group. They found the increase in the 
Black-White achievement gap was largest for the students who in Grade 3 had the highest levels 
of reading. By comparison, for students with the lowest levels of reading in Grade 3, the increase 
in the Black-White achievement gap from Grade 5 to Grade 8 was smaller or negative. 


Similarly, when Clotfelter, Ladd, and Vigdor (2009) examined changes over time as the same 
group of students advanced through school, they found that racial achievement gaps between low- 
performing students tended to get smaller as students progressed through school, while the racial 
achievement gaps between high-performing students increased. The gaps were between White 
students and American Indian, Black, and Hispanic students. The study’s sample was students who 
attended North Carolina public schools in Grade 3 and for five years subsequently, including 
students who repeated a grade. 


In their later study, Plucker, Hardesty, and Burroughs (2013b) provided profiles of the gap in 
achievement at the advanced level for each state. In Maryland, relatively few students had 
advanced scores on NAEP for 2003, 2007, or 2011; the highest rate was 20% among Grade 4 
students not receiving FARMS on reading in 2011. Nonetheless, based on NAEP scores, there 
were gaps of 5—16 percentage points every year in the excellence gap in favor of White vs. Black 
students, and White vs. Hispanic students, as well as non-ELL vs. ELL students and non-recipients 
vs. recipients of FARMS. For the Maryland School Assessments (MSA), these authors presented 
data for Grades 4 and 8 in reading and math for 2004—2011 plus high school reading and Algebra 
data for the same years, when available. There were double-digit gaps every year in favor of White 
vs. Black students, White vs. Hispanic students, and non-recipients vs. recipients of FARMS. 


Study in MCPS The Office of Legislative Oversight for Montgomery County analyzed 
changes in achievement gaps among MCPS students from 2007 to 2012 and from 2010 to 2012 
(Bonner-Tompkins, Richards, & Scruggs, 2013). This study examined racial/ethnic gaps, along 
with gaps related to receipt of the following services: English for Speakers of Other Languages 
(ESOL), Free and Reduced-price Meals System (FARMS), and special education. Their findings 
indicated that MCPS narrowed the achievement gaps on three of four grade-level measures: school 
readiness, MSA proficiency in reading and math, and graduation rate, and also on two of three at- 
risk measures: suspension rates and academic ineligibility in middle school. However, the 
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achievement gap widened for four measures that the Office of Legislative Oversight identified as 
above-grade-level: 


Advanced level on MSA in Grades 3, 5, and 8 

Algebra I by Grade 8 with C or higher 

Score of 3 or higher on AP exam or score of 4 or higher on IB exam 
SAT 1650 or higher/ACT 24 or higher 


Advanced Level Performance of Black or African American and Hispanic/Latino Students 


Recommended strategies and programs Ford (2004) proposed a model or framework to 
address the needs of students who are gifted and culturally diverse. The framework recommends 
modifying the content, process, product, and learning environment to deliver gifted education and 
also to deliver multicultural education (see Table 1). 


Table 1 
Ford’s Framework to Address Educational Needs of Students who are Gifted and Culturally Diverse 
Gifted education Multicultural education 

Content Content is modified through the use of | Content is modified to include greater focus on 
more advanced or complex concepts, multicultural concepts, issues, themes, events, 
abstractions, and materials. and persons. Curricular resources and materials 

are multicultural. 

Process Activities are redesigned or Instructional strategies, including teaching 
restructured to be more intellectually | styles, are modified to match more closely the 
demanding. Higher level thinking, learning and cognitive styles of culturally 
problem solving, inquiry-based diverse students. Students’ cultural 
learning are used, as well as backgrounds and characteristics are given 
acceleration. substantive consideration in instructional 

practices. 

Product Students share their learning in varied | Students share their learning in many ways, but 
ways, particularly by producing a multicultural focus is always present to some 
products that are authentic, address degree. Students develop products that address 
real issues, and have real audiences. issues and solve problems germane to culturally 

diverse populations. 

Learning Teachers create environments that are | Teachers create learning environments that 

environment | safe, non-judgmental, and student- affirm students’ identity as cultural beings; 
centered such that students are at ease | teachers use the cultural characteristics of 
in taking intellectual and creative diverse groups to create a learning community 
risks. Teachers create a sense of where all individuals are valued and respected. 
community in which all gifts and 
talents are valued and respected. 


Briggs and Reis (2004) conducted case studies of seven gifted education programs, to determine 
their success with groups of culturally, linguistically, and ethnically diverse students. Two 
programs targeted high school students. Project College Bound (in the Los Angeles Unified 
School District) focused on a group of culturally diverse students from low income families. While 
in Grades 10-12, students received information and support to help them pursue college 
opportunities. The program also involved parents and local high school college counselors. 
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Almost all students in the first graduating group enrolled in college. A second program, the Mentor 
Connection (on the University of Connecticut Storrs campus), gave juniors and seniors an 
opportunity to explore a professional interest for three weeks. Special efforts were made to include 
a culturally diverse group. The program concluded that about 30% of participants subsequently 
attended the University of Connecticut due to the Mentor Connection. 


Olszewski-Kubilius and Clarenbach (2012) focused on low-income, high-achieving students and 
stressed that strategies to support students impacted by poverty differed from those to support 
students of different racial/ethnic groups. Based on examinations of several programs that targeted 
low-income, high ability students and enabled more of them to improve their academic 
achievement and success in school, the authors identified the following best practices: 


1) Gateway programs. Programs focused on preparation of students for advanced courses or 
programs at the next level of schooling to equip them to make these critical transitions. 

2) Program selection criteria matched to level of developed talent. Selection criteria included 
multiple measures, did not use cutoff scores, and relied on holistic assessments to identify 
students who were both interested in and committed to achievement. 

3) High-powered curriculum. Courses were challenging and enriched. Teachers provided 
scaffolding for advanced thinking and questioning skills, not remediation. Professional 
development of teachers was extensive and focused on changing teacher attitudes and 
expectations away from a deficit viewpoint. 

4) Significantly expanded learning time. Students spent more time in classes, usually outside 
of the normal school day, such as after school, on weekends, and during summer or other 
breaks. 

5) Providing program components that equalize opportunities. Programs provided services 
or connections that more affluent families could access, such as tutors, private college 
counselors, and contact with adults who could offer internships or career information. 

6) Augmenting student support networks. Students were in classes with others who were 
smart and talented and could provide peer support. Some programs also assigned adult 
mentors or included parent education components. 


Studies in MCPS. One study identified MCPS middle schools with sustained success in 
narrowing the gap between Asian American and White students and African American and 
Hispanic students (Cooper-Martin, 2012). The goal was to identify school-level policies, 
strategies, and practices that helped to close the achievement gap. Some, although not all, of the 
data points in this study reflected achievement at an advanced level in reading or mathematics. 
Members of the teaching staff at every middle school completed an online survey. Survey items 
with responses that differed significantly between middle schools with sustained success in 
narrowing the gap and the remaining middle schools were identified. These items related to one 
of the following three themes: 1) school leadership for learning, teaching, and equity; 2) teacher 
support for use of data; and 3) school culture. 


A second study explored the postsecondary institution activity of African American and Hispanic 
graduates of MCPS from 2001 to 2010 (Addison-Scott & Maina, 2012). Reflecting both Hoover’s 
Vindicationist Philosophy (1990) and Harper’s Anti-Deficit Achievement Framework (2010), the 
study focused on achievement as opposed to “gaps.” Overall, 76% of the African American and 
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65% of the Hispanic 2001—2010 MCPS graduates enrolled in a postsecondary institution at some 
time after graduation. Among African American or Hispanic students who graduated in 2008 or 
later and who enrolled in college, nearly all enrolled in college within one year after high school 
graduation. Among older graduates (i.e., the MCPS classes of 2001 to 2006), 30% (3,837) earned 
a bachelor’s degree or higher within six years of high school graduation, and close to one half of 
those who had not finished within six years had earned their degree by May 2011. Among younger 
graduates (i.e., the MCPS classes of 2007 to 2010), more than 8 out of 10 were enrolled as full- 
time students as of May 2011. 


Program Evaluation Unit 9 Achievement at an Advanced Level, Secondary 


Scope of the Study 


The goal of this study was to explore achievement at the advanced level, with a focus on identifying 
those programs and curricular options that supported advanced levels of performance by Black or 
African American and Hispanic/Latino students as they went on to complete high school. 


Research Questions 
1. What is the level of achievement at the advanced level among secondary students over time? 


a. What percentage of students performed at an advanced level in Grade 8? 

b. Among students who performed at an advanced level in Grade 8, what proportion 
continued as high achievers at the end of high school? 

c. Which students were most likely to stay at the advanced level over time, from Grade 8 
to the end of high school? 


2. Which high school programs or curricular options were most promising in supporting Black 
or African American and Hispanic/Latino students to continue achievement at an advanced 
level at the end of high school? 


a. In which programs or options were high-achieving Black or African American and 
Hispanic/Latino Grade 8 students most likely to continue as high achievers at the end 
of high school? 

b. Which programs or options had the most impact on supporting Black or African 
American and Hispanic/Latino Grade 8 students to continue as high achievers at the 
end of high school? 


3. With respect to the programs and curricular options identified in question 2, 


a. What are the characteristics or components of each program or option? What are 
common aspects across the programs and curricular options? 

b. How many students participated by demographic and service subgroups in each 
program or curricular option? 

c. How did Black or African American and Hispanic/Latino students in each of the 
programs or curricular options perform, compared to their White and Asian peers in 
that program or option, on state and district performance measures? 
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Methodology 
Study Sample 


The study sample was a cohort of students who completed Grade 8 in MCPS in the 2008-2009 
school year. 


Measures of Achievement 


Grade 8. To identify Grade 8 students achieving at an advanced level, the study used 
measures of both reading and mathematics. Program staff helped determine appropriate measures. 
One measure was performance on the Grade 8 Maryland School Assessments (MSA) in 
mathematics; specifically, whether students achieved an advanced level. Although MSA scores 
have become less aligned with the curriculum and thus less useful since 2012—2013, when MCPS 
introduced the Common Core curriculum, the students in the study sample were 8" graders in 
2008-2009. Thus, when these students were in Grade 8, instruction was aligned to the MSA tests 
and it is appropriate to use their MSA test results. A second measure of achievement at the 
advanced level was performance in a mathematics course, specifically, earning a grade of B or 
higher (both semesters) in Algebra 1 (or a higher level math course) in Grade 8. The third measure, 
in reading, was a score of 231 or higher on the Measures of Academic Progress assessments in 
reading (MAP-R) (Zhao, 2011). In summary, there were three indicators of high achievement in 
Grade 8 (Table 2). A student only needed to meet one of them to be identified as a high achiever 
in Grade 8. 


Table 2 
Measures of Student Achievement by Grade Level 
Grade level & districtwide milestones Data point for high achievement* 
Grade 8: 
e Mathematics at the advanced level e Advanced level on MSA mathematics test 
e Reading at the advanced level e Mathematics course (Algebra I or higher) 
with a grade of B or higher in Grade 8 
e MAP-R score of 231 or higher 
Grade 12 
e SAT 1500 or higher (out of 2400) or ACT 21 or e SAT 1650 or higher (out of 2400) or ACT 24 or 
higher higher 
e Score of 3 or higher on AP exam or Score of 4 or Score of 3 or higher on AP exam or Score of 4 
higher on IB exam or higher on IB exam 
e On-time graduation e Completion of a rigorous high school program 
(as defined by the Maryland State Department of 
Education) 


*See text for definitions and details. 


Grade 12. When the students in the study graduated from MCPS, the districtwide 
milestone for performance on SAT or ACT tests was 1650 or higher (out of 2400) on the SAT and 
24 composite score for the ACT. Although MCPS lowered the milestone for the SAT and ACT 
during fall 2015 (Lang, 2015), the earlier one was used; it was higher, which seemed more 
appropriate for a study of high achievement, and was in effect when these students graduated. 
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The second districtwide milestone for performance by the end of Grade 12 is a score of 3 or higher 
on an AP exam or a score of 4 or higher on IB exam. This milestone was used as a measure of 
high achievement because the Office of Legislative Oversight for Montgomery County identified 
it as above grade level, along with the 1650 SAT/24 ACT milestone, in their analysis of student 
achievement (Bonner-Tompkins, Richards, & Scruggs, 2013). 


The third districtwide milestone for performance by the end of Grade 12 is on-time graduation. 
As a measure of high achievement that was similar to this milestone, this study used completion 
of a rigorous high school program, using the following definition from the Maryland State 
Department of Education (MSDE, 2014): a student who has mastered four of the following six 
indicators has completed a rigorous high school program: 


e Two or more credits in the same foreign language with a grade of B or higher 

e One or more credits in mathematics courses at a level higher than Algebra 2 and Geometry 
with a grade of B or higher 

e Four credits of science with a grade of B or higher 

e Two or more credits of approved advanced technology education’ with a grade of B or 
higher 

e Ascore of 1000 or higher on SAT-1 or a score of 20 or higher on ACT, or both 

e Acumulative GPA of 3.0 or higher on a 4.0 scale 


In summary, there were three indicators of high achievement by the end of Grade 12 (Table 2 
above). Students only needed to meet one of them to be identified as a high achiever at the end of 
high school: 


e SAT 1650 or higher (out of 2400) or ACT 24 or higher 

e Score of 3 or higher on AP exam or Score of 4 or higher on IB exam 

e Completion of a rigorous high school program as defined by MSDE (see previous 
paragraph) 


Data Sources 


Student-level information from appropriate MCPS databases was downloaded to provide the 
measures for high achievement, program codes, course completion records, and demographic and 
other descriptors about students to answer research questions 1 and 2. Program information on the 
MCPS website was the source for descriptions of each program to answer question 3a. 


Measures of Programs and Curricular Options 


Program codes from the Online Administrative Student Information System (OASIS) data base 
were used to identify student participation in magnet, choice, or signature programs. Because 
some schools did not provide any codes and because no participants were identified in OASIS for 
some programs, additional approaches were necessary, as follows. Students who completed the 


> Advanced technology education courses allow students to develop in-depth skills and knowledge related to one or 
more of the following areas — impacts of technology, technological issues, engineering design (MSDE, 2016). 
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IB test for the Theory of Knowledge course while enrolled in MCPS were identified as enrolled in 
the IB program, along with those identified by program codes from OASIS. Analysis of course 
completion was used to identify enrollment in the following seven programs: APEX at Walter 
Johnson High School, APPS at Clarksburg High School, Biomedical Magnet Program at Wheaton 
High School, Engineering Magnet Program at Wheaton High School, Humanities & Arts at 
Wootton High School, PEAC Scholars at Col. Zadok Magruder High School, and technology at 
Thomas Edison High School of Technology. Lastly, no enrollees could be identified from the 
available program codes and course histories for the following programs: Communication House 
at Poolesville High School; Gifted and Talented/Learning Disabled at Walter Johnson, Northwood, 
or Watkins Mill High School; and Visual Art Center at Albert Einstein High School. Therefore, 
there are no results for these programs. 


Also, the complexity of analysis required to identify student participation in magnet, choice, or 
signature programs was so time consuming that it precluded identification of students outside of 
the study sample (e.g., Asian or White students). Therefore, there are no findings for questions 3b 
and 3c. 


Records of course completion were used to identify student participation at the end of Grade 12 in 
four types of courses that support advanced learning: Advanced-level, AP, Honors, and IB (see 
definitions above). Records of course completion also were used to identify student participation 
in the dual enrollment program; because only one student in the study sample participated, there 
are no results for dual enrollment. 


Analytical Procedures 


In calculating whether a student met an advanced level indicator on Algebra | by Grade 8, any 
student who did not take the course was categorized as not meeting the milestone. Further, a 
student who did not take any AP or IB exams was categorized as not meeting the advanced level 
indicator for that measure; all MCPS high schools offer these courses, so all students have the 
opportunity to take these courses and thus be eligible to take the exam. (The data on AP or IB 
exams did not indicate whether students had taken the corresponding course.) Likewise, for the 
SAT/ACT measure, a student who did not take either of the tests was categorized as not meeting 
the advanced level. This approach, of including students who did not take the relevant course, 
exam, or test, permitted the inclusion of a greater number of students. Lastly, students who failed 
to graduate were categorized as not meeting any of the advanced level measures for Grade 12. 


Descriptive statistics were used to analyze the findings for the following research questions: 
questions la and 1b on the percentage of students who performed at an advanced level in 
Grade 8 and then continued as high achievers at the end of high school; question 2a on those 
programs or options in which high-achieving students were most likely to continue as high 
achievers; and question 3a on common aspects across the programs and curricular options 
identified in question 2. 


For research question Ic on changes over time, descriptive statistics were used. To determine 
which groups of students were most likely to stay at the advanced level over time, a y” test was 
used to test for statistically significant differences between groups of students. For each 7’ test, 
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the degrees of freedom, which equals the number of student groups minus one, is reported along 
with the number of students, represented as N. For example, in an analysis of differences between 
genders, the number of categories would be two and so the degrees of freedom equals one (i.e., 
two minus one), as in the following: y°(1, N = 3,351) = 6.98. 


To answer research question 2b on the impact of programs or curricular options, y° analyses were 
used to test whether enrollees in each program or option were statistically more likely than non- 
enrollees to perform at an advanced level at the end of high school. Non-enrollees were limited to 
students who also had the opportunity to enroll in that program or curricular option, as included in 
the descriptions above. For example, non-enrollees were limited to students in the same school 
for signature programs and to schools with IB programs for the curricular option of enrollment in 
IB courses, while all students in the sample were included for country-wide programs. (See more 
details on non-enrollees in the footnotes of Table 4.4, below.) For some programs or curricular 
options, the number of enrollees was much smaller than the number of non-enrollees. However, 
unequal groups are not a problem for y” (Greenwood & Nikulin, 1996). 


Effect sizes (i.e., Phi) from the y’ analyses were calculated and compared across high school 
programs and curricular options to identify which programs or options had the most impact on 
supporting high-achieving students to continue as high achievers. Effect sizes were appropriate 
for comparisons between programs and curricular options because these measures are metric free 
and as a result were not affected by the scale of the variables included in the analyses. Effect sizes 
are tests of practical significance (American Psychological Association, 2001). Phi was interpreted 
as follows: .10 for a small effect, .30 for a medium effect, .50 for a large effect (Nandy, 2012). 


For each of the four types of courses that support advanced learning, chi-square analyses were 
repeated for a range of semesters (e.g., 4, 6, 8), because students could take more than one year of 
them. Further, the median number of semesters completed varied across courses from 16 for 
Honors to 0 for IB (Appendix A). The enrollment level with the largest impact on supporting 
high-achieving students to continue as high achievers (as indicated by the largest biggest Phi) was 
included in the findings for question 2b. 


Strengths and Limitations of the Methodology 


A strength of this study is the use of analysis of course histories and other approaches to confirm 
program enrollment along with the codes available in OASIS; this approach resulted in including 
more enrollees and more programs. However, the difficulty of analyzing course histories made it 
too time-consuming to identify students outside of the subgroup of interest (i.e., Black or African 
American and Hispanic/Latino) and thus prevented answering questions 3b and 3c. Further, 
because these additional approaches relied on analysis of all four years of high school to identify 
students, they could not be used to confirm program enrollment for Grade 9. Therefore, although 
there was an interest in answering research questions 2a and 2b about programs and options that 
support advanced achievement at the end of Grade 9, it was not possible to conduct the relevant 
analyses. Also, despite the use of additional approaches to identification, it was not possible to 
identify any students for some programs and so the study could not include those programs. 
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The chi-square tests used to answer question 2b did not control for any differences between 
enrollees and non-enrollees in each program or curricular option. This lack of controls means that 
any differences in achievement between enrollees and non-enrollees may be due to other 
differences between the two groups, rather than due to the program or curricular option. However, 
the enrollees and non-enrollees were similar in race/ethnicity, grade level, achievement level, and, 
for some programs, the middle school or high school attended. 


In this study of students over time (as in most longitudinal studies), the sample decreased from 
Grade 8 to Grade 12, as students dropped out or moved out of MCPS. These changes could affect 
the results, if there was a pattern to which students left MCPS. Lastly, it should be noted that 
causality may not be inferred from the findings of this study due to the lack of an experimental 
design. 
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Results 


Question 1: What is the Level of Achievement at the Advanced Level Among Secondary 
Students Over Time? 


Grade 8. Table 3.1 presents the proportion of students who performed at an advanced level 
in Grade 8 on each of the three measures or on any of them. The one measure in reading was the 
score on the MAP-R test; about one half of students (51%) performed at the advanced level on this 
measure. Fewer students achieved at an advanced level on either of the mathematics measure; 
about one out of four students scored at the advanced level based on the MSA test (39%) and on 
their mathematics course grade (42%). Based on any of the three measures, nearly one out of six 
(59%) Grade 8 students performed at an advanced level in 2008-2009. 


Table 3.1 
Proportion of Students in Grade 8 who Performed at the Advanced Level on Three Measures 
Advanced in Grade 8 on 
MSA Mathematics | At least one 
MAP-R mathematics course grade measure 

Student characteristics and Total Nin | (V=9,851)? (N=10,401)° (N=10,601) | (NV =10,601) 
service receipt in Grade 8 Grade 8 n % n % n % n % 
All students 10,601 | 4,981 50.6 4,022 38.7 4,396 41.5 | 6,292 59.4 
Gender 

Female 5,153 | 2,588 53.8 2,017 39.9 2,364 45.9 | 3,216 62.4 

Male 5,448 | 2,393 47.5 2,005 37.5 2,032 37.3 | 3,076 56.5 
Race/ethnicity 

American Indian 30 15 57.7 8 27.6 11 36.7 18 60.0 

Asian 1,562 | 985 66.2 976 63.3 1,026 65.7 | 1,249 80.0 

Black or African American 2,501 730 32.1 391 16.0 495 19.8 960 38.4 

Hispanic/Latino 2,293 516 24.1 348 15.8 436 19.0) 774 33.8 

White 4,215 | 2,735 69.7 2,299 55.4 2,428 57.6 | 3,291 78.1 
ESOL services groups 

Never received 7,830 | 4,215 58.0 3,349 43.6 3,555 45.4 )]5,114 65.3 

Past not current 2,240 | 735 34.3 612 27.7 753 33.6 | 1,070 47.8 

Current recipient 531 31 8366.9 61 12.1 88 16.6 108 20.3 
FARMS services groups 

Never received 6,360 | 3,954 66.9 3,350 53.4 3,566 56.1 | 4,826 75.9 

Past not current 1,303 445 36.1 Zoe 2A 317 24.3 577 44.3 

Current recipient 2,938 582 21.5 381 13.4 513 17.5 889 30.3 
Special education services groups 

Not current recipient 9,263 | 4,765 55.1 3,889 42.5 4,242 45.8 | 5,993 64.7 

Current recipient 1,338 216 17.9 133 10.7 154. 11.5 BB | 


*750 students were missing scores. 
200 students were missing scores. 


With respect to subgroups of students, slightly more female than male students performed at an 
advanced level on each measure. Among racial/ethnic groups, higher percentages of American 
Indian, Asian, and White students than Black or African American and Hispanic/Latino students 
achieved advanced levels on each measure. For each of the services groups, smaller percentages 
of current recipients than other groups performed at the advanced level on every measure. 
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Grade 8 to Grade 12. Among the group of 6,292 students who performed at an advanced 
level in Grade 8, over 5,700 of them (91%) were still enrolled in MCPS in 2012-2013 or had 
already graduated. The majority of these students still performed at an advanced level of 
achievement in Grade 12 (Table 3.2). Specifically, about two thirds (66%) performed at an 
advanced level on AP/IB exams, close to two thirds (64%) had SAT/ACT scores at the advanced 
level, and close to six out of ten (59%) completed a high school program at the advanced level. 


Table 3.2 
Students Advanced in Grade 8 and Advanced in Grade 12 on Three Measures, 
by Demographic Characteristics and Service Receipt 


Advanced in Grade 12 on 
Total NV AP/IB SAT/ACT High school 
advanced exams scores program completion | Any measure 

Student characteristics and in (N = 5,622)* (N = 5,708) (N = 5,708) (N = 5,708) 
service receipt in Grade 8 Grade 8 n %o n %o n Jo n %o 
All students 5,708 3,717 66.1 3,656 64.1 3,352 58.7 4,349 76.2 
Gender 

Female 2,936 1,985 68.4 1,874 63.8 1,877 63.9 2,298 78.3 

Male Cee 32 OS 1,782 64.3 1,475 pS 2,051 74.0 
Race/ethnicity? 

Asian 1,177 921 79.3 891 75.7 886 75.3 1,030 87.5 

Black or African American 826 332 = 40.2 272 32.9 256 31.0 409 49.5 

Hispanic/Latino 695 302.) 43.5 201 28.9 209 30.1 2,552 85.3 

White 2,993 2,155 = 73.7 2,286 76.4 1,994 66.6 350 850.4 
ESOL services groups 

Never received 4,651 3,111 68.0 3,177 68.3 2,843 61.1 3,656 78.6 

Past not current 984 564 557.8 452 45.9 472 48.0 643 65.3 

Current recipient 73 42 60.9 27 = =37.0 37 50.7 50 68.5 
FARMS services groups 

Never received 4,384 3,170 = 73.5 3256 74.3 2,925 66.7 3,684 84.0 

Past not current 546 251 47.0 YO BID) 203 37.2 308 56.4 

Current recipient 778 296 =. 38.2 193 24.8 224 28.8 357 = 45.9 
Special education services groups 

Not current recipient 5,438 3,622 67.5 3,525 64.8 3,260 59.9 4,201 77.3 

Current recipient 270 95 37.1 131 48.5 92 34.1 148 54.8 


Note: Excludes students who withdrew prior to 2012-2013. Includes early graduates. 
* 86 students were missing data. 
>Excludes American Indians due to small sample size. 


More likely to stay at advanced. Several subgroups of students were more likely to stay 
at the advanced level over time, from Grade 8 to the end of high school. A significantly higher 
percentage of female than male students performed at an advanced level on two measures: AP/IB 
exams (68% vs. 64%, y°(1, N = 5,622) 14.3, p < .0 1) and high school program completion (64% 
vs. 53%, ¥2(1, N = 5,708) = 67.6, p < .001) in Grade 12. 


On each Grade 12 measure, more students who were Asian or White stayed at an advanced level 
over time than Black or African American students or Hispanic/Latino students. For example, on 
AP/IB exams, 79% of Asian students and 74% of White students performed at an advanced level 
at the end of high school, compared to 40% of Black or African American students and 44% of 
Hispanic/Latino students. 
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Additional calculations (not reported in Table 3.2) were used to test for statistical significance in 
differences between racial/ethnic groups. Among students who were Asian or White, 75% stayed 
at an advanced level based on AP/IB exams compared to 42% of students who were Black or 
African American or Hispanic/Latino; this difference was statistically significant (y7(1, N = 5,622) 
= 562.6, p < .001). The differences between these groups were significant for the other two 
measures. Specifically, among students who were Asian, or White, 76% stayed at an advanced 
level over time based on SAT/ACT scores compared to 31% for Black or African American or 
Hispanic/Latino; this difference was statistically significant (y7(1, N = 5,708) = 990.0, p < .001). 
Likewise, 69% of students who were Asian or White stayed at an advanced level over time, based 
on high school program completion, compared to 31% of other racial/ethnic groups; this difference 
was statistically significant (y7(1, N = 5,708) = 682.7, p < .OO1). 


Among ESOL service groups, a significantly higher percentage of Grade 8 students who had never 
received ESOL services stayed at an advanced level over time as measured by AP/IB exams (68%) 
than students who were past not current recipients (58%) and students who were current recipients 
(61%) (¥7(2, N = 5,622) = 37.5, p < .001). This pattern of significant differences also occurred for 
SAT/ACT scores (¥7(2, N = 5,708) = 200.1, p < .001), and high school program completion (¥7(2, 
N = 5,708) = 60.0, p < .001). 


Among FARMS service groups, significantly more Grade 8 students who had never received 
FARMS stayed at an advanced level over time as measured by AP/IB exams (74%) than students 
who were past not current recipients (47%) and students who were current recipients (38%) (7(2, 
N = 5,622) = 461.7, p < .001). This pattern of significant differences was repeated for SAT/ACT 
scores (¥7(2, N = 5,708) = 881.2, p < .001) and high school program completion (y?(2, N = 5,708) 
= 507.8, p < .001). 


Lastly, with respect to special education services, significantly more Grade 8 students who were 
not currently receiving special education services than Grade 8 students who were currently 
receiving such services stayed at an advanced level over time on all three measures: AP/IB exams 
(68% vs. 37%, 71, N = 5,622) = 100.7, p < .001), SAT/ACT scores (65% vs. 49%, 71, N = 
5,708) = 29.7, p < .001), and high school program completion (60% vs. 34%, ¥°(1, N = 5,708) = 
71.0, p < .001). 


In summary, several subgroups of students were more likely to stay at the advanced level from 
Grade 8 to the end of high school: females, Asian or White students, students who had never 
received ESOL, students who had never received FARMS, and non-recipients of special education 
services. In response to the finding that Black or African American and Hispanic/Latino students 
were less likely to continue at an advanced level at the end of high school, the rest of this study 
focused on those students. 
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Question 2: Which Programs and Curricular Options Supported Continued Achievement 
by Black or African American and Hispanic/Latino students at an Advanced Level? 


The sample for question 2 was smaller than that for question 1; it was limited to the 1,521 Black 
or African American and Hispanic/Latino students who were high achievers in Grade 8 in the 
2008-2009 school year and who also completed Grade 12 in MCPS. 


Question 2a: Most likely to continue achievement at advanced levels in high school. The 
first set of findings about programs or curricular options in which students were most likely to 
continue as high achievers concerns high school programs. For each of 11 high school programs, 
the percentage of program enrollees who performed at the advanced level at the end of high school 
is in Table 4.1. Note that two programs at Poolesville, Global Ecology (n=8) and Humanities 
(n=7), were combined to ensure a large enough size for analysis. The programs in which enrollees 
were most likely to continue as high achievers were: CAP at Montgomery Blair HS, IB at Richard 
Montgomery HS; Science, Mathematics, & Computer Science at Montgomery Blair HS and 
Poolesville HS; and Global Ecology or Humanities House at Poolesville HS. In these four 
programs, all or almost all of the enrollees continued as high achievers at the end of Grade 12. 


Table 4.1 
Proportion of Black or African American and Hispanic/Latino Students who Were High 
Achievers in Grade 8 and who Performed at the Advanced Level at the End of High School 


by High School Program 
At advanced level at 
Total | end of high school 

High school program number n %o 
CAP at Montgomery Blair HS 11 11 100.0 
IB program at Richard Montgomery HS 17 17 100.0 
Science, Mathematics, & Computer Science at Montgomery 

Blair HS and Poolesville HS 10 10 100.0 
Global Ecology or Humanities House at Poolesville HS 15 14 93.3 
Local IB programs 98 81 82.7 
Leadership Training Institute at John F. Kennedy HS 9 7 77.8 
Biomedical Magnet Program at Wheaton HS 20 11 55.0 
Engineering Magnet Program at Wheaton HS 12 6 50.0 
Technology at Thomas Edison High School of Technology 30 > 16.7 


The second set of results about programs or curricular options in which students were most likely 
to continue as high achievers concerns signature programs, which are unique to one MCPS high 
school, and thus have few students. Therefore, the following four signature programs that each 
require AP courses were combined to ensure a large enough size for analysis: APEX Program at 
Walter Johnson High School (n=3), APPS at Clarksburg High School (n=6), Humanities & Arts 
(H&A) at Wootton High School (n=4), and PEAC Scholars at Col. Zadok Magruder High School 
(n=8). Although the two signature programs at Winston Churchill High School had only four 
enrollees, they were not combined with the others because they do not require AP courses. The 
number of enrollees in the Ulysses Program at Northwest HS was large enough that this signature 
program was analyzed by itself. 
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For each signature program, Table 4.2 shows the percentage of enrollees who performed at the 
advanced level at the end of high school. The programs in which enrollees were most likely to 
continue as high achievers were the group of four signature programs that require AP courses 
(APEX, APPS, H&A, PEAC) and the Ulysses Program at Northwest High School. In these 
signature programs, all of the enrollees performed at the advanced level at the end of Grade 12. 


Table 4.2 
Proportion of Black or African American and Hispanic/Latino High Achievers in Grade 8 who 
Performed at the Advanced Level at the End of High School by High School Signature Program 


At advanced level at 
Total | end of high school 
High school signature program number n %o 
Four programs that require AP courses (APEX, APPS, H&A, PEAC) 21 21 100.0 
Two programs at Winston Churchill HS 4 4 100.0 
Ulysses Program at Northwest HS Di DD 81.5 


The third set of results about programs or curricular options in which students were most likely to 
continue as high achievers concerns curricular options, specifically enrollment in the following 
four types of courses that support advanced learning. Advanced-level courses are based upon 
previous achievement in a sequence of study. AP courses are those courses for which a College 
Board Advanced Placement examination exists. An Honors course includes curriculum 
adaptations for accelerated and enriched learning. IB courses are part of the IB curriculum; 
students at six high schools with the IB program can enroll in IB courses without enrolling in the 
full program. 


Table 4.3 shows the percentage of enrollees in two or more semesters (i.e., equivalent to one year) 
of each type of course who performed at the advanced level at the end of high school. The 
curricular option in which enrollees were most likely to continue as high achievers was IB courses. 
Over two-thirds of the 237 students who completed two or more semesters of an IB course 
performed at the advanced level at the end of Grade 12, based on any measure. 


Table 4.3 
Proportion of Black or African American and Hispanic/Latino High Achievers in Grade 8 who Performed 
at the Advanced Level at the End of High School by Four Types of Courses (N = 1,521) 


At advanced level at end of 
# of enrollees in two high school 
Type of course or more semesters n %o 
Advanced level 1,290 732 56.7 
AP 1,254 744 59.3 
Honors 1,478 746 50.5 
IB 237 163 68.8 


Question 2b: Most impact. Effect sizes were calculated to show if the impact of each 
program or curricular option on supporting high-achieving students to continue as high achievers 
was practically significant. Chi-square analyses were calculated to test whether enrollees in each 
program or option were statistically more likely than non-enrollees (see definitions in footnotes of 
Table 4.4) to continue as high achievers at the end of high school. Phi, an effect size, was 
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calculated for each chi-square and interpreted as follows: .10 for a small effect, .30 for a medium 
effect, .50 for a large effect. Chi-square analyses were repeated for each curricular option for a 
range of semesters (e.g., 4, 6, 8), because students could take more than one year of these courses. 
The enrollment level for each curricular option with the largest effect size (i.e., largest Phi) is 
presented in Table 4.4, along with the results for each program. 


Table 4.4 
Proportion of Black or African American and Hispanic/Latino High Achievers in Grade 8 (N=1,521) 
who Performed at the Advanced Level at the End of High School 
by Enrollment in High School Programs and Curricular Options 


Enrollees Non-enrollees 
Advanced at end Advanced at end 
Total Of high school | Tot} of high school 2 

High school program or curricular option # n % # n % (df=1) Phi 
Six or more semesters of AP courses 822 623 75.8 699" =136 19.5 479. 5*** 56 
Six or more semesters of Advanced-level courses 721 530 135) 800? 229 28.6 305.6*** 45 
Ulysses Signature Program at Northwest HS 27 22 81.5 62> 22 35,5 15.9*** 42 
Four or more semesters of IB courses 164 130 79.3 sway 13} 34.8 86.3*** 42 
Local IB programs 98 81 82.7 3294 122 37.1 62.9*** 38 
Four signature programs that require AP courses 
(APEX, APPS, H&A, PEAC) 21 21 ~=100.0 203° 81 39.9 PED Tpaeo esis) 
CAP at Montgomery Blair HS 11 11 100 425' 172 40.5 15.6*** 19 
Ten or more semesters of Honors courses 1,244 671 53.9 277? = 88 31.8 AMAL Speers IT 
IB program at Richard Montgomery HS 17 17. —-100.0 |1,504* 689 45.8 19.8*** 11 
Global Ecology or Humanities House at Poolesville 
HS 15 14 93.3 |1,506* 692 45.9 13.4*** 09 
Science, Mathematics, & Computer Science at 
Montgomery Blair HS and Poolesville HS 10 10 =100.0 |1,511* 696 46.1 11.6*** 09 
Biomedical Magnet Program at Wheaton HS 20 11 55.0 416! 187 45.0 0.8 .04 
Engineering Magnet Program at Wheaton HS 12 6 50.0 4246 177 41.7 0.3 03 
Technology at Thomas Edison HS 30 5 16.7 |1,491*° 701 47.0 10.9*** -.09 
Leadership Training Institute at John F. Kennedy HS 9 7 77.8 427) 191 44.7 NA NA 
Two signature programs at Winston Churchill HS 4 4 100.0 26° 16 61.5 NA NA 


Note. Programs and curricular options are not mutually exclusive. 

“Includes all other students. 

> Includes only students who attended Northwest HS in Grade 12. 

“Includes only students who, in Grade 12, attended eight high schools that offer an IB program. 

‘Includes only students who, in Grade 12, attended seven high schools that offer a local IB program. 

© Includes only students who, in Grade 12, attended Clarksburg, Magruder, Walter Johnson, or Wootton high school. 
Includes only students who attended a DCC middle school in Grade 8. 

® Includes only students who attended Churchill HS in Grade 12. 

NA= not available, because the sample size is too small for y’. 

*p < 05, **p <. 01, ***p <. 001 


The option of enrolling in six or more semesters of AP courses had the largest Phi and thus the 
greatest impact on supporting high-achieving students to continue performing at an advanced level 
(Table 4.4 above). It was the only program or curricular option with a large effect size 
(Phi =.58), meaning that the observed relationship between enrollment in this curricular option and 
continued high achievement was practically significant and large. The curricular options or 


Program Evaluation Unit 21 Achievement at an Advanced Level, Secondary 


programs with the next-largest impact had medium effect sizes (i.e., Phi > .30) and also were 
practically significant. There were five options or programs with medium effect sizes: six or more 
semesters of Advanced-level courses, Ulysses Signature Program, four or more semesters of IB 
courses, local IB programs, and signature programs that require AP courses. 


Note that several programs or curricular options had a Phi below .10, meaning that any difference 
between enrollees and non-enrollees was not practically significant. Also, note that the programs 
and curricular options examined were not mutually exclusive. For example, the option of six or 
more semesters of AP courses included students in the Ulysses Signature Program and those in the 
signature programs that required AP courses. Similarly, the option of four or more semesters of 
IB courses included students in local IB programs and those in the Richard Montgomery IB 
program. 


Summary. Ten programs and curricular options (out of 19 analyzed) were identified as 
the most promising in supporting high-achieving, Grade 8 Black or African American and 
Hispanic/Latino students to continue achievement at an advanced level. They included four high 
school programs in which all or nearly all participants finished high school at an advanced level: 


CAP at Montgomery Blair HS 

Global Ecology or Humanities House at Poolesville HS 

IB at Richard Montgomery HS 

Science, Mathematics, & Computer Science at Montgomery Blair HS and Poolesville HS 


The remaining six promising programs and curricular options were those with the greatest impact 
(i.e., the biggest effect size as measured by Phi) on supporting high-achieving students to continue 
as high achievers at the end of high school: 


Six or more semesters of Advanced-level courses 

Six or more semesters of AP courses 

Four or more semesters of IB courses 

Four signature programs that require AP courses (APEX, APPS, H&A, PEAC) 
Local IB programs at seven high schools 

Ulysses Signature Program at Northwest HS 
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Question 3: Characteristics of Programs and Curricular Options that Supported Continued 
Achievement at an Advanced Level 


This section provides more information on the 10 programs and curricular options for advanced 
learning that were identified in question 2 as the most promising in supporting high-achieving, 
Grade 8 Black or African American and Hispanic/Latino students to continue achievement at an 
advanced level. 


Descriptions. Appendix B presents descriptions for each of the ten promising programs 
and curricular options. 


Common aspects. All 10 of the promising programs or curricular options required 
enrolling in at least one of the four types of courses that support advanced learning, as follows: 


e AP for five programs or options: CAP at Montgomery Blair HS, six or more semesters of 
AP courses, Global Ecology or Humanities House at Poolesville HS, the four signature 
programs that require AP courses (APEX, APPS, H&A, PEAC), Ulysses Signature 
Program at Northwest HS. 

e IB for three programs or options: four or more semesters of IB courses, IB at Richard 
Montgomery HS, local IB programs. 

e Advanced-level for two programs or options: six or more semesters of Advanced-level 
courses; Science, Mathematics, & Computer Science at Montgomery Blair HS and 
Poolesville HS. 


Six of the ten promising programs and curricular options required applications: 


e CAP at Montgomery Blair HS 

e Four signature programs that require AP courses (APEX, APPS, H&A, PEAC) 

Global Ecology or Humanities House at Poolesville HS 

IB at Richard Montgomery HS 

Science, Mathematics, & Computer Science at Montgomery Blair HS and Poolesville HS 
Ulysses Signature Program at Northwest HS 


Four or five of the promising programs and curricular options required a research project during 
the student’s junior or senior year, specifically: 


CAP at Montgomery Blair HS 

IB at Richard Montgomery HS 

Local IB programs 

Two (i.e., APEX, H&A) of the four signature programs that require AP courses 
Ulysses Signature Program at Northwest HS 


There was not a common length of time among the promising programs and options. The three 
curricular options could occur anytime during a student’s high school years, both IB programs 
were focused on two years (junior and senior), and the remaining five programs involved all four 
years of high school. 
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Discussion 


Seven of the promising programs and curricular options involved enrollment in AP or IB courses, 
which is a pre-requisite to taking an AP or IB exam. Because scores on such exams were one of 
the three measures used to identify advanced achievement, there may be concerns due to the 
similarity between these definitions and the measure of success. Therefore, additional analysis 
that excluded performance on AP or IB exams was conducted; it used completing a rigorous high 
school program (as defined by MSDE) as the only measure of advanced achievement at the end of 
high school. It was used instead of ACT/SAT performance because it is more comprehensive. 


The first analysis concerned the four programs in which all or almost all students continued as 
high achievers at the end of high school, when using any of three measures (Table 4.1 above). 
When using only one measure for high achievement (i.e., completion of a rigorous high school 
program), the percentage of enrollees who continued as high achievers remained at or close to 
100% for CAP at Montgomery Blair HS and for Science, Mathematics, & Computer Science at 
Montgomery Blair HS and Poolesville HS (Table 5). However, for the other two programs, Global 
Ecology or Humanities House at Poolesville HS and IB at Richard Montgomery HS, the 
percentage of enrollees who were advanced at the end of high school declined to about 70% when 
using only one measure for advanced achievement at the end of high school (i.e., completion of a 
rigorous high school program), compared to more than 90%, when using the three measures of 
advanced achievement at the end of high school (i.e., SAT/ACT, AP exam/IB exam, or completion 
of a rigorous high school program). 


Table 5 
Proportion of Black or African American and Hispanic/Latino High Achievers in Grade 8 (N=1,521) 
who Completed a Rigorous High School Program by Programs and Curricular Options 
Enrollees 


Completed 
rigorous high 
school program 


High school program or curricular option Total # n % 

Science, Mathematics, & Computer Science at Montgomery Blair HS & Poolesville HS 10 10 100.0 
CAP at Montgomery Blair HS 11 10 90.9 
Four signature programs that require AP courses (APEX, APPS, H&A, PEAC) 21 17 81.0 
Two signature programs at Winston Churchill HS 4 3 75.0 
Global Ecology or Humanities House at Poolesville HS 15 11 73.3 
IB program at Richard Montgomery HS 17 12 70.6 
Leadership Training Institute at John F. Kennedy HS 9 6 66.7 
Ulysses Signature Program at Northwest HS 27 15 55.6 
Local IB programs 98 48 49.0 
Six or more semesters of AP courses 1,037 445 42.9 
Four or more semesters of IB courses 164 70 42.7 
Six or more semesters of Advanced-level courses 721 355 49.2 
Biomedical Magnet Program at Wheaton HS 20 8 40.0 
Ten or more semesters of Honors courses 1,244 420 33.8 
Engineering Magnet Program at Wheaton HS 12 4 33.3 
Technology at Thomas Edison HS 30 2 6.7 
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The second analysis involved chi-square tests of whether enrollees in any of the programs or 
curricular options were statistically more likely than non-enrollees (see definitions in footnotes of 
Table 4) to complete a rigorous high school program. Based on the values of Phi from these 
analyses, there was no change in which six programs and curricular options had the most impact 
on supporting high achievers (Table 6). Although there were variations in the values of Phi for 
the top six programs or curricular options compared to the values from the analysis using all three 
measures for advanced achievement (see Table 4.4 above), the effect size for all six was medium. 


Table 6 
Proportion of Black or African American and Hispanic/Latino High Achievers in Grade 8 (N=1,521) who 
Completed a Rigorous High School Program by Enrollment in High School Programs and Curricular Options 


Enrollees Non-enrollees : 
Phi for 
Completed Completed advanced 
rigorous high rigorous high on at 
Total School program | Tot¢q] School program Me least one 
High school program or curricular option # n % # n % (df=1) Phi | measure 
Six or more semesters of AP courses 822 403 49.0 699" 62 8.9 287.0*%** 43 56 
Six or more semesters of Advanced-level 
courses 721 = 355 49.2 |} 800* 110 Bish || Za Oe 8h) 45 
Ulysses Program at Northwest HS 27 15 55.6 62> 12 19.4 11.7*** 36 42 
Four or more semesters of IB courses 164 70 42.7 3252S) 16.3 Al seney i) 42 
Local IB programs 98 48 49.0 3294 58 17.6 39.8*** 31 38 
Four signature programs that require AP 
courses (APEX, APPS, H&A, PEAC) 21 17 81.0 203° 50 24.6 23) SO 35 
Ten or more semesters of Honors courses 1,244 420 33.8 277° 45 16.2 32.85") ~ 315 17 
IB program at Richard Montgomery HS 17 2 70.6 |1,504* 453 30.1 13.0*** 09 11 
Global Ecology or Humanities House at 
Poolesville HS 15 11 73.3 |1,506* 454 30.1 13.1*** 09 .09 
Science, Mathematics, & Computer Science 
at Montgomery Blair HS and Poolesville HS 10 10 100.0 /1,511*% 455 30.1 DD) See) 09 
Biomedical Magnet Program at Wheaton HS 20 8 40.0 4166 117 28.1 1.3 .06 04 
Engineering Magnet Program at Wheaton HS 12 4 33.3 424° 121 28.5, NA NA NA 
Technology at Thomas Edison HS 30 2 6.7 |1,491* 463 31.1 8.2** — -.07 -.09 
CAP at Montgomery Blair HS 11 10 90.9 ADs NS) D7 NA NA NA 
Leadership Training Institute at Kennedy HS 9 6 66.7 427, 119 27.9 NA NA NA 
Two signature programs at Churchill HS 4 3 75.0 26® 10 38.5 NA NA NA 


Note. Phi for advanced on any measure taken from Table 2.4 above. 


“Includes all other students. 


> Includes only students who attended Northwest HS in Grade 12. 
“Includes only students who, in Grade 12, attended eight high schools that offer an IB program. 
‘Includes only students who, in Grade 12, attended seven high schools that offer a local IB program. 
Includes only students who, in Grade 12, attended Clarksburg, Magruder, Walter Johnson, or Wootton high school. 
Includes only students who attended a DCC middle school in Grade 8. 
= Includes only students who attended Churchill HS in Grade 12. 
NA= not available, because the sample size is too small for y’. 


*p < 05, **p <. O1, ***p <. 001 


From this re-analysis, two programs, Global Ecology or Humanities House at Poolesville HS and 
IB at Richard Montgomery HS, appeared less promising for supporting continued high 
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achievement when performance on AP/IB exams was not used as a measure of high achievement, 
compared to the results when using any of three measures (including AP/IB exams). However, 
the students in IB at Richard Montgomery HS also were included in the curricular option of four 
or more semesters of IB courses; that curricular option continued to be one of those with the most 
impact on continued high achievement, even when excluding the AP/IB exams measure. In 
conclusion, with the exception of the two Houses at Poolesville, the identification of which 
programs and curricular options were the most promising was not an artifact of the similarity 
between the program’s requirements (i.e., enrollment in AP or IB courses) and the measure of 
success (i.e., scores on AP or IB exams). 


Recommendations 


The report’s findings suggest the following recommendations to support continued achievement 
of Black or African American and Hispanic/Latino students: 


e Starting in middle school, encourage Black or African American and Hispanic/Latino 
students who are performing at an advanced level to enroll in at least two, year-long 
Advanced-level, AP, or IB courses in high school. Augment course bulletins to indicate 
which courses are Advanced-level. 

e Starting in middle school, encourage Black or African American and Hispanic/Latino 
students who are performing at an advanced level to enroll in IB programs or signature 
programs that require AP courses, at high schools with these programs. 

e Consider establishing a signature program or modifying an existing signature program that 
targets advanced-level learners so that the program requires AP courses at the following 
high schools that do not have such a program and do not have an IB program: 


o James Hubert Blake o Northwood 

o Winston Churchill o Paint Branch 

o Damascus © Quince Orchard 
o Thomas Edison o Sherwood 

o Gaithersburg o Walt Whitman 


e Consider expanding the availability of Advanced-level, AP, and IB courses and the size of 
IB and signature programs to support increased enrollment in these programs and curricular 
options by Black or African American and Hispanic/Latino students who performed at an 
advanced level in Grade 8. 


The difficulties in identifying student participation in programs precluded identification of students 
outside of the study sample, of student participation in these programs as ninth graders, and of any 
students for certain programs. As a result, the study’s analyses and findings were limited. The 
final recommendations on future research include one to avoid these difficulties: 


e Develop a monitoring system to ensure that staff members at every school consistently and 
completely enter program codes for participants for every year of the program. 

e Confirm the study’s findings by replicating the research with a more recent class of MCPS 
graduates. 
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Appendix A 


Table A 
Statistics for Number of Semesters Completed by Black or African American and 
Hispanic/Latino High Achievers in Grade 8 (N = 1,521) for Four Types of Courses 
Semesters by the end of Grade 12 


Advanced- AP Honors IB 
level courses courses courses courses 
Mean 7.0 6.6 15.0 1.7 
Median 5.0 6.0 16.0 0.0 
Standard deviation 7.6 5.3 5.8 5.3 
Minimum 0 0 0 0 
Maximum 44 24 32 31 
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Appendix B 


Descriptions of Nine Promising Programs and Curricular Options that Supported 
Continued Achievement at an Advanced Level 


Communication Arts Program (CAP) at Montgomery Blair HS 

The Communication Arts Program is an application-only, four year, honors-level program with 
an interdisciplinary approach designed for students who wish to develop their intellectual, 
creative, and communication skills in the fields of media and the humanities. CAP is unique 
among secondary programs in MCPS because of the strong interdisciplinary connections 
among the humanities courses. These connections have led to an emphasis on student-centered, 
performance-based assessments, as each project reflects a variety of content areas. Once 
admitted to the program, students must successfully complete 13 CAP courses, maintain a 
minimum 2.75 GPA, and complete the CAP portfolio to graduate with the CAP certificate. 
Students follow a nine-period schedule which extends the school day until 3:20. (Description 
retrieved Oct. 3, 2016 http://cap.mbhs.edu/newdocs/CAP%20Brochure %202016.pdf.) 


Six or more semesters of Advanced-level courses 
Advanced-level courses are based upon previous achievement in a sequence of study; these 
courses are available in several content areas, including information technology/computer 
science, foreign languages, mathematics, and science. 


Six or more semesters of AP courses 
AP courses are those courses for which a College Board Advanced Placement examination 
exists. 


Four or more semesters of IB courses 
IB courses are offered as part of the IB program (see description below). 


Four signature programs that require AP courses (APEX, APPS, H&A, PEAC) 
1) The APEX Scholars program [at Walter Johnson HS] is designed as a four year course of 
study through which a select group of students follows a comprehensive curriculum combining 
accelerated honors level and intensive advanced placement level course work. The course of 
study provides students with a broad, general background in many academic areas at levels 
appropriate to highly academically gifted and motivated learners. (Description retrieved Aug. 
20; 2016 http://www.montgomeryschoolsmd.org/schools/wjhs/programs/ 


apex/program_info.aspx.) 


2) The APPS Program is designed as a four year course of study through which students follow 
a comprehensive curriculum combining honors level and advanced placement level course 
work. Students accepted into the program will follow a rigorous course of study designed to 
challenge the highly motivated learner. Students will receive structured support and skill 
refinement through their honors and advanced placement courses as well as through periodic 
lectures and seminars held at school. The program is designed to offer students flexibility in 
their schedules as they progress though their four years of study and allow them to pursue 
individual educational goals while at Clarksburg High School. (Description retrieved Sept. 8, 
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2016 http://www.montgomeryschoolsmd.org/curriculum/specialprograms _/high/signatures. 
aspx?id=141241.) 


3) The Humanities and Arts (H&A) Signature Program ...[at Wootton HS] is designed for 
students who are passionate about literature, film, history, languages, art, and/or performance. 
The students form a cohort in the 9th-10th grades through special enriched sections of English 
and Social Studies, which include special field trips, speakers, and artists-in-residence 
workshops. Students select a specific but flexible course of study with the Humanities or Arts 
Focus, and also complete an Off-Campus Experience and the Senior Independent Project. By 
the end of the program, students have polished oral presentation skills and are able to complete 
the high-level academic research and writing expected by universities. (Description retrieved 


Sept. 8, 2016  http://(www.montgomeryschoolsmd.org/curriculum/specialprograms/high/ 
signatures.aspx?id=141265 


4) PEAC Scholars [at Col. Zadok Magruder HS] PEAC is an inter-disciplinary, four-year 
college-preparatory cohort. Its focus is on independent learning and the development of 
sophisticated written and oral communication skills with an emphasis on critical thinking. 
Designed for the highly able student, PEAC Scholars offers accelerated courses that are only 
available to those accepted into the program. (Description retrieved Aug. 25, 2016 


http://www.montgomeryschoolsmd.org/schools/magruderhs/peac/.) 


Global Ecology or Humanities House at Poolesville HS 
The Global Ecology House provides students with the knowledge and understanding of the 
science and cultural, social, political, economic, and technological conditions that affect the 
quality of life on our planet. Students will get a broad background of ecological information 
from which they may pursue a variety of environmental perspectives. The program also offers 
a unique computer-assisted approach to the study of environmental science and social studies, 
including networking with students and schools, locally and around the globe. Students will be 
challenged to develop a global ecological perspective as they investigate historical and current 
issues related to the earth’s natural resources. (Description retrieved Sept. 8, 2016 
http://(www.montgomeryschoolsmd.org/curriculum/specialprograms/high/ magnet- 


ecology.aspx.) 


Humanities House students will pursue a rigorous humanities curricula with an 
interdisciplinary connection among English, social studies, communications, and fine arts. 
Humanities students will be able to integrate oral and written communication skills with state- 
of-the-art visual media. (Description retrieved Sept. 8, 2016 http://www. 


montgomeryschoolsmd.org/curriculum/specialprograms/high/magnet-humanities.aspx.) 


IB at Richard Montgomery HS and Local IB programs at seven high schools 
The International Baccalaureate Diploma Program is a demanding two-year international 
program that meets the needs of highly motivated Grade 11 and 12 students and leads to a 
qualification that is recognized by leading universities around the world. To earn an IB 
diploma, students must take a challenging liberal arts course of studies and pass examinations 
in six academic subjects. In addition, students are required to take the Theory of Knowledge 
(TOK) course that investigates the nature of knowledge in various disciplines; participate in 


i) 
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Creativity, Action, Service (CAS) to achieve eight learning outcomes, undertake original 
research, and write an Extended Essay of 4,000 words. Students successfully completing the 
program earn a MCPS diploma and a Montgomery County Certificate of Merit, in addition to 
the internationally recognized IB diploma. (Description retrieved Aug. 25, 2016 
http://www.montgomeryschoolsmd.org/curriculum/specialprograms/high/ib.aspx.) 


Science, Mathematics, & Computer Science at Montgomery Blair HS and Poolesville HS 

The Science, Mathematics, Computer Science Magnet program is designed to offer accelerated 
instruction in the areas of math, science, and computer science for highly motivated and able 
students. It is a rigorous program that emphasizes the development of problem-solving skills, 
critical thinking, and the pursuit of both independent and collaborative research projects. 
Students will have unique opportunities to partner with mentors from many local and national 
businesses and _ research organizations. (Description retrieved Sept. 8, 2016 
http://(www.montgomeryschoolsmd.org/curriculum/specialprograms /high/magnet-science.aspx. ) 


Ulysses Signature Program at Northwest HS 

The Ulysses Program is a four-year program that encourages student-initiated explorations in 
areas of strong interest. Research skills are emphasized through signature sections of English, 
history, and science. All course offerings are at the honors or AP level; interdisciplinary 
connections are encouraged. Through independent and collaborative research and technology- 
infused presentations, students experience various modes of inquiry while learning to use their 
minds well. Ulysses students study, create, and achieve in an intellectual and nurturing 
community of learners. (Description retrieved Aug. 25, 2016  http://northwesths- 
montgomeryschoolsmd.libguides.com/Ulysses.) 
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